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Abstract 



We study the positive recurrence of piecewise Ornstein-Uhlenbeck (OU) diffusion 
processes, which arise from many-server queueing systems with phase-type service re- 
quirements. These diffusion processes exhibit different behavior in two regions of the 
Ph ■ state space, corresponding to 'overload' (service demand exceeds capacity) and 'under- 

Ph I load' (service capacity exceeds demand). The two regimes cause standard techniques 

(~| ■ for proving positive recurrence to fail. Using and extending the framework of com- 

mon quadratic Lyapunov functions from the theory of control, we construct Lyapunov 
functions for the diffusion approximations corresponding to systems with and without 
abandonment. With these Lyapunov functions, we prove that piecewise OU processes 
have a unique stationary distribution. 

(N. 

1^ . Keywords: stability, common quadratic Lyapunov function, Lyapunov function, piece- 

wise OU process, multi-server queues, customer abandonment, Halfin-Whitt regime, phase- 

00 ■ type distribution. 
(N 

1 1 Introduction 



Since the pioneering paper of iHalfin and WhittI ( 198ll ). and particularly within the last 10 



years, there has been a surge of interest in diffusion approximations for queueing systems 
k> . with many servers. These quei ieing systems mode l customer contact centers with hundreds 



of servers. Empirical study in iBrown et al.l ( 20051 ) suggests that the service time distribu- 



tion is far from exponential. Despite past and foreseeable advances in computer hardware 
and architectures, the sheer size of such systems prohibits exact (numerical) calculations 
even when the arrival process is Poisson and the service time distribution is of phase type. 
Diffusion approximations such as piecewise Ornstein-Uhlenbeck (OU) processes can be used 
to approximate the queue len gth process. Such approximat ions are rooted i n man y-server 
heavy traffic limits proved in iPuhalskii and ReimanI (2000|) and iDai et al.l (2010|). These 



approximations are remarkably accurate in pred icting systern perfo rmance measures, some- 
times for systems with as few as 20 servers, see iHe and Dail ( 201ll ). 



For a diffusion approximation to work, it is critical to know whether the approximating 
diffusion process has a unique stationary distribution. In this paper we prove that, under 
some natural conditions, this is the case fo r piecewise OU pr ocesses. Thus, this paper 
provides a solid mathematical foundation for iHe and Pail ( 201ll ). who devise an algorithm 



to numerically compute the stationary distribution of a piecewise OU process. 

A standard technique for proving stability of queueing systems is to first establish the 
stability of a so-called fluid model and then to app e al to g enera l theo r y for est a blishin g 
stochastic stability (see, e.g., IPupuis and Williams! ()1994l ): IPail (119951 ): IStolyan (jl995l )). 



However, this theory is restricted to systems with nonnegative fluid levels which are at- 
tracted to the origin. The fluid analog of a piecewise Ornstein-Uhlenbeck process does 
not possess this property. As an alternative to the fluid model framework, the family of 
quadratic Lyapunov functions is a natural choice for establishing positive recurrence. In- 
deed, due to diffusive properties of piecewise Ornstein-Uhlenbeck processes, if a quadratic 
Lyapunov function can be shown to stabilize the fluid model, it simultaneously and directly 
establishes stochastic stability, i.e., the positive recurrence of piecewise OU processes. As 
a result of working with quadratic forms as Lyapunov functions, several key results from 
linear algebra lie at the heart of our main results. We were unable to devise an equally 
powerful framework without using this algebraic machinery. 

Piecewise OU processes exhibit different behavior in two regions of the state space, cor- 
responding to 'overload' and 'underload'. The two regions are separated by a hyperplane, 
which corresponds to 'critical load'. In each of the two regions, a piecewise OU process can 
be thought of as a first-order linear differential equation with stochastic noise. A standard 
technique in proving its positive recurrence is to use a quadratic Lyapunov function to 
prove stability of such first-order linear differential equations. However, the two different 
regions of a piecewise OU process pose considerable challenges to apply this methodology. 
A natural approach would be to 'paste together' two quadratic Lyapunov functions from 
the two regions, but our attempts in this direction have failed. In fact, it is we ll-known that 



a diffu sion with two stable regim es can lead to an instable hyb rid system, see lYin and Zhu 



S) for related examples. In E^;^ and TsitsikliJ S), the stability of a switched 



linear system is discussed from the perspective of complexity theory. 

Using the interpretation of the diffusion parameters in terms of a many-server queueing 
system, our main results can be formulated as follows: (1) For a slightly underloaded system 
without abandonment, we show that there exists a quadratic Lyapunov function which 
yields the desired positive recurrence using the Foster-Lyapunov criterion (Theorem [2]) . In 
general, this quadratic Lyapunov function is not explicit and non-unique. (2) We show 
that no quadratic Lyapunov function can satisfy the Foster-Lyapunov criterion for systems 
with abandonment. (3) We construct a suitable non-quadratic Lyapunov function to prove 
positive recurrence for systems with abandonment (Theorem [3|) . 

The main building blocks for these two types of Lyapunov functions are so-called com- 
mon quadratic Lyapunov functions (CQLFs), which are widely used in the theory of con- 
trol. Such functions play an important role in the stability analysis for deterministic linear 



systems, with different dynamics in different parts of tlie state space (or, more generally, 
operating under a switching rule). They are called common quadratic Lyapunov functions 
since they serve as a quadratic Lyapunov function in each part of the state space. There 



i s a v ast body of literature on CQLFs and related theory, see the survey [Shorten et al 
((20071) for details. Althou gh quadratic Lyapunov functions are ubiquitous in the liter - 



l|zuu/i ) tor aetans. Aitnou gn quaaratic .Lyapunov lunctions are ubiquitous m tne nter - 
ature on queueing systems JDai and Prabhakarl ( 200d ): iGamarnik and Momcilovid ( 20081 ): 



Tassiulas and EphremidesI (jl992l ). to our knowledge, our paper is the first to exploit CQLFs 
in this context. 



As mentioned in the section on open problems of lShorten et al.l (J2007l ). it is of consider- 
able interest to determine simple conditions for the existence of CQLFs. Theorem (H which 
is our main technical contribution in this space, establishes such a result in the context 
of M-matrices and rank-1 perturbations. The theorem shows that existence of a CQLF is 
guaranteed after merely verifying that certain vectors are nonnegative. It is a first result of 
this kind. Its proof relies on a delic ate analysis involving Cheb yshev polynomials, a.s wel l 
as on an extension of recent work of lKing and NathansonI ( 20061 ) and lShorten et al.l ( 2003 ) 
summarized in Proposition [3] below. 

To conclude this introduction, we mention a body of work on the rec urrence of multidi- 
mensional Ornstein-Uhlenbeck type processes bv lSato et al.l ( 1994l . ll996l ). which differ from 
the processes studied here. The processes studied in these references are driven by Levy 
processes and they have a linear drift coefficient. As a result, their multidimensional pro- 
cesses do not possess the critical feature of the processes we study here, namely a piecewise 
linear drift coefficient. 

This paper is organized as follows. Section 2 discusses the required background on 
piecewise OU process and positive recurrence. Section 3 is devoted to common quadratic 
Lyapunov functions. Section 4 summarizes the main results and Section 5 contains the 
proofs of the main results. The proof of Proposition [3l which mainly uses existing method- 
ology from the theory of control, is given in Appendix A. Appendix B shows that no 
quadratic Lyapunov function can work in the Foster-Lyapunov criterion if abandonment is 
allowed. 



Notation 

All random variables and stochastic processes are defined on a common probability space 
{Q,J-,F) unless otherwise specified. For some d € N, M'^ denotes the d-dimensional Eu- 
clidean space. The space of functions / : M — )■ M that are twice continuously differentiable 
is denoted by C^(M^). We use V to denote the gradient operator. Given x G M, we set 
x"*" = max{x, 0}. All vectors are envisioned as column vectors. For a i^T-dimensional vector 
u, we use Uk to denote its fcth entry and we write \u\ for its Euclidean norm. We also write 
u' for its transpose. For two i^-dimensional vectors u and v, we write u' > v' [u' > v') if 
Uk > Vk {uk > Vk) for each k = 1,2, ... ,K. The inner product of u and v is denoted by 
u'v, which is '^p.^iUkV^- Given a. K x K matrix M, we use M' to denote its transpose 



and Mij for its {i,j) th entry. We write M > (M < 0) if M is a positive (negative) 
definite matrix and M > (M < 0) if it is a positive (negative) semi-definite matrix. Let 
the matrix norm of M be \M\ = '^■- \Mij\, where \Mij\ is the absolute value of Mij. We 
reserve / for the K x K identity matrix and e for the i^-dimensional vector of ones. 

2 Piecewise OU processes and positive recurrence 

This section introduces the piecewise Ornstein-Uhlenbeck (OU) processes studied in this 
paper, and discusses preliminaries on positive recurrence. 

2.1 Piecewise OU processes 

We first define M-matrices. We call a matrix nonnegative when each element of the matrix 
is nonnegative. 

Definition 1 (M-matrix). A K x K matrix R is said to be an M-matrix if it can he 
expressed as R = si — N for some s > and some nonnegative matrix N with the property 
that p{N) < s, where p{N) is the spectral radius of N. The matrix R is nonsingular if 
p{N) < s. 

We next define piecewise Ornstein-Uhlenbeck (OU) processes, which are special dif- 
fusion processes. Let {Vr(t)} be a standard Brownian motion in any dimension. A K- 
dimensional diffusion process Y is the strong solution to a stochastic differential equation 
of the form 

dY{t) = h{Y{t))dt + a{Y{t))dW{t), 

where the drift coefficient 6(-) and the diffusion coefficient a{) have appropriate sizes and 
satisfy the following Lipschitz continuity condition: there exists some C > such that 

\b{x)-h{y)\ + \cr{x)-a{y)\<C\x-y\ for all x,y G M^"^. (2.1) 

For a real- valued function V G C^(M^), the generator G of y applied to V is given by, for 

1 d'^V 

GV{y) = {VV{y))'h{y) + - ^^ (^^'), .(y)-^(y). (2.2) 



We refer to ( Rogers and Williams! . 120001 . Chapter V) for more details on diffusion processes. 



Definition 2 (Piecewise OU processes). Let p he a K -dimensional prohahility vector, e 
he the K -dimensional vector of ones, and let R he a K x K nonsingular M-matrix. For 
a, /3 G M, a K -dimensional diffusion process Y is called a piecewise Ornstein- Uhlenheck 
(O U) process if it has drift coefficient 

b{y) = -Pp - R{y - p{e'y)+) - ap{e'y)+, (2.3) 



and diffusion coefficient a{y) = a for all y G R^, such that a a' is a K x K nonsingular 
matrix. 



As in IPai et al.l ( 2010l ). we call this process a piecewise OU process since the drift 



coefficient is affine (hence OU process) yet it differs on each side of the hyperplane {y G M^^ : 
e'y = 0} (hence piecewise). Indeed, for e'y > we have b{y) = —(3p — R{I — pe')y — ap{e'y) 
while for e'y < we have b{y) = —(ip — Ry. In conjunction with o"(y) = u, this implies 
the Lipschitz continuity condition (j2.ip . As a consequence, the piecewise OU process Y is 
well-defined as a diffusion process. 

The quantities a, /3, R,p on the right-hand side of (j2.3p come from the queueing system 
that gave rise to the piecewise OU diffusion. Their queueing interpretation is as follows: 
a is the abandonment rate, /3 is the slack in the arrival rate relative to a critically loaded 
system, while p and R are the parameters of the service- time distribution (assumed to be 
of phase-type). For more details, we refer to lDai et al.l ( 2010 ). 



Throughout the paper, we impose the following assumption. 

Assumption 1. Each component of the row vector e'R is nonnegative, i.e., 

e'R > 0'. 

We now make the connection between piecewise OU processes and many-server queue- 
ing models explicit, and we discuss Assumption [1] in this context. For presentational 
convenience, we do so in the special case of M/H2/n + M queues. In fact, we consider 
a sequence of M/ H2/n -\- M queues indexed by n, where n is the number of (identical) 
servers, meaning that (1) the arrival process is a Poisson process with some intensity A", (2) 
the service times have a two-phase hyperexponential distribution, so they are exponential 
with parameter vi with probability pi, and exponential with parameter V2 with probabil- 
ity P2 = 1 — Pi; and (3) each customer has a patience time which follows an exponential 
distribution with parameter a > 0. Hypergeometric service time d istributions are of spe- 



cial in t erest, since they can be used t o model multiclass systems, see lPuhalskii and Reiman 



( 20001 ) : iGamarnik and Stolvaii ( 201ll ). To see this, envision two types of customers entering 



a buffer to seek service. Suppose that the mean service time is 1, i.e., pi/vi +P2/V2 = 1- 
We further assume the system is operated under Halfin-Whitt regime, i.e., for some /3 G M, 

lim ^ f 1 - — ) = /3. 

n^oo y n J 

Let X'^{t) and ^2*(t) denote the number of customers of type 1 and 2 in the system at 
time t. For i = 1,2 and i > 0, we define 

X^(t) = ^(xnt)-n^t 
\/n V Ui 



As detailed in iDai et alj ( 2010 ) (see iGamarnik and Goldberg] ( 201ll ) for a related general 
result), the 'centering' in this expression has been chosen so that, in a sense of weak 
convergence on the process level, 

where (Yi, I2) satisfies the following system of stochastic differential equations: for i = 1,2, 
Y,{t) = Yi{0) + W^{t)-f3p^t 

-V, [ {Yi{s) - p^iYiis) +Y2{s))+)ds - ap^ [ {Yi{s) + Y2is))+ds. 
Jo Jo 

Note that {Yi{s) + l2(s))^ represents the (scaled) number of customers waiting in the 
buffer, and the fraction of type i customers in the buffer is approximately pi. Thus, the 
term involving Vi can be thought of as a service-rate term. Similarly, the terms involving a 
and /3 are the abandonment and arrival term, respectively. The randomness in the system 
is represented hy W = {Wi,W2), which is a driftless Brownian motion with nonsingular 
covariance matrix 

( PiiPic^ - Pi + 2) PiP2{c^ - 1) \ 

V P1P2{C^-1) P2{P2C^-P2 + '2) J' 

for some constant c € M. Therefore, Y = {Yi,Y2) is a two-dimensional piecewise OU 
process with drift coefficient 

b{y) = -Pp - R{y - p{e'y)+) - ap{e'y)+, (2.4) 



where the matrix R is given by 



R 



^1 





1^2 



When we apply this procedure to a general phase-type service time distribution with K 
phases, the corresponding diffusion limit is a ET-dimensional piecewise OU process Y . The 
parameters p and R represent the distribution of the initial phase and phase dynamics, 
respectively. Each component of the piecewise OU process Y^ approximates the number of 
phase-A; customers in the many-server queueing system, either waiting or in service. Thus 
e'Y represents the total number of customers in the system after centering and scaling. 
Thus, whenever e'Y > the system is in "overload", i.e., there are customers waiting in 
the buffer, and whei iever e'Y < the syste n i is in "un derload", i. e ., the re are idle servers. 
We refer readers to IPuhalskii and ReimanI (2000) and I Dai et al.l ( 20ld ) for more details. 
We remark that the matrix R in these two papers takes the form of {I — P') diagjz^}, where 
P is assumed to be a transient matrix describing the transitions between each service 
phase, and diagji^} is a diagonal matrix with fc-th diagonal entry given by Vk, where Vk is 
the rate for the sojourn time in phase k. Transience of P corresponds to customers who 
eventually leave the system after receiving a sufficient amount of service, which implies 
that e'R = e'{I — P') diagjz^} > 0. Therefore, we conclude that in this setting i? is a 
nonsingular M-matrix and that Assumption 1 is satisfied. 
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2.2 Positive recurrence and Lyapunov functions 

In this section, we recall the definitions and the criteria for positive recurrence and expo- 
nential ergodicity in the context of general diffusion processes. 

Let E^ be the expectation operator with respect to a probability distribution vr. 

Definition 3 (Positive recurrence and stationary distribution). For a K -dimensional dif- 
fusion process Y, we say that Y is positive recurrent if for any y G R-^' and any compact 
set C in M with positive Lebesgue measure, we have 

E{tc\Y{0) = y)<oo, 

where tq = inf{t > : Y{t) G C} is the hitting time of the set C . We call a probability dis- 
tribution vr on M a stationary distribution for Y if for every bounded continuous function 
f: M^^ -^ M, 

E^[/(y(t))] = E,[/(y(0))] for all t > 0. 

In the following, we assume that the diffusion coefficient of the diffusion process Y is 
uniformly nonsingular. That is, there exists some c € (0, oo) such that for all y G M and 

a'a{y)a{yya > ca'a. (2-5) 

The next result giv es a sufficient criterion for posi tive recurrence of diffusion processes, 



see (jKhasminskiil . |20 1 ll . Section 3.7, 4.3 and 4.4), and (iMevn and Tweediel.ll993l. Se ction 4). 
Uniqueness of the stationary distribution follows from iPeszat and ZabczykI ()1995l ) in view 
of condition (12.51). 



Proposition 1 (Foster-Lyapunov criterion). Let Y be a diffusion process satisfying i2.5\) . 
Suppose that there exists a nonnegative function V G C^(M ) and some r > such that, 
for any \y\ > r, 

GV{y) < -1. 

In addition, suppose that V{y) —t- oo as \y\ — t- oo. Then Y is positive recurrent and has a 
unique stationary distribution. The function V is called a Lyapunov function. 

We now introduce the concept of exponential ergodicity. For any positive measurable 
function / > 1 and any signed measure m, we write \\m\\f = sup|g|<j \in{g)\. 

Definition 4 (Exponential ergodicity). Suppose that the diffusion process Y is positive 
recurrent and that it has a unique stationary distribution vr. Given a function f > 1, we 
say that Y is f -exponentially ergodic if there exists a 7 G (0, 1) and a real-valued function 
B such that for all t > and y G M.^ , 

\\P\y,-) -7r{-)\\ f<B{y)^', 

where P* is the transition function ofY.Iff = l,we simply say that Y is exponentially 
ergodic. 
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For / > 1, we have ||P*(y, •) — vr(-)||i < ||P*(y, •) - vr(-)||/, and we deduce that /- 
exponential ergodicity imphes exponential ergodicity. Th e following result gives a criterion 
for exponential ergodicity, see ( Mevn and Tweedid . Il993l . Section 6). 



Proposition 2. Suppose that Y is a diffusion process with a unique stationary distribution. 
If there is a nonnegative function V G C^(M ) such that V{y) —)• oo as |y| —t- oo and for 
some c > 0,d < oo, 

GV{y) < -cV{y) + d for any y G R^ , 

then Y is {V + 1)- exponentially ergodic. 

3 Common quadratic Lyapunov functions 

In this section we introduce common quadratic Lyapunov functions (CQLFs). Such func- 
tions play a central role in the stability analysis of deterministic switched linear systems, 
which is discussed in Section [3.21 We use CQLFs as building blocks to construct Lyapunov 
functions to prove positive recurrence of piecewise OU processes. At this point it is best to 
distinguish CQLFs for switched linear systems from the Lyapunov functions in the context 
of the Foster-Lyapunov criterion. We connect these two concepts in Section HI 

3.1 Background and definitions 

Quadratic Lyapunov functions form a cornerstone of stability theory for ordinary differen- 
tial equations. Consider the linear system y{t) = By{t) where y{t) G M , B G M is 
a fixed real matrix and y{t) is the derivative of y with respect to t. For Q G M^^^, the 
quadratic form L given by L{y) = y'Qy for y G M^^ is called a quadratic Lyapunov function 
for the matrix i? if Q is positive definite and QB + B'Q is negative definite. In this case, 
there exists a constant C > such that 

j^L{y{t)) = y{t)'{QB + B'Q)y{t) < -CL{y{t)) < for aU t > 0, 

and thus we can conclude that L{y{t)) < e~ L(y(0)). This implies that L{y(t)) — t- as 
t — )• oo, thus y(t) — ;• as t — ;■ oo. It is standard fact in Lyapunov stability theory that the 
existence of a qua dratic Lyapunov function L is equivalent to all eigenvalues of B having 
negative real part ( Berman and Plemmond . Il994l . Section 6.2). 



The following definition, tailored to our setting in order to allow for a singular matrix 



plays an im portant role in our an alysis. Other versions can be found in lShorten and Narendra 



(120031 ) and IShorten et al.l (J2007l ) . Recall that an eigenvalue of a matrix is called (geomet 



rically) simple if its corresponding eigenspace is one-dimensional. 

Definition 5 (CQLF). Let Bi G M have all eigenvalues with negative real part and let 

i?2 G M^^^ have all eigenvalues with negative real part except for a simple zero eigenvalue. 



For Q G M. , the quadratic form L given by L{y) = y'Qy for y G R is called a common 
quadratic Lyapunov function (CQLF) for the pair {Bi,B2) if Q is positive definite and 

QBi + B[Q < 0, 
QB2 + B'2Q < 0. 



3.2 The CQLF existence problem 

The CQLF existence problem for a pair of matrices has its roots in the study of stabiHty 
criteria for switched hnear systems. These systems have the form y{t) = B{T)y{t) where 
B(t) S {i?i,i?2} with Bi E M^^^ for z = 1,2 and where the switching function r may 
depend on both y and t. The existence of a CQLF for the pair {Bi,B2) guarantees that 
all solutions of the systems are bounded under arbitrary switching function r. The CQLF 
existence problem is also closely related to the Kalman-Yacubovich-Popov lemma in the 
development of adaptive control algorithms and the Lur'e problem in nonlinear feedback 
analysis. For more detail s consu lt iKalmanI ( 19631 ). iBoyd et al.l ( 19941 ) and the recent survey 



paper by [Shorten et al 



( 20071 ) . For an arbitrary matrix pair, no simple analytic and 
verifiable conditions are known for the pai r to admit a CQLF. I n the special case where 
the difference of the matrices has rank one. King and NathansonI (120061 ) shows that if both 
Bi and B2 are Hurwitz, i.e., all eigenvalues of the matrices Bi,B2 have negative real part, 
then there exists a positive definite matrix Q such that QBi+B[Q < and QB2 + B2Q < 
if and only if the matrix product B1B2 has no real negative eigenvalues. Note that in this 
ca se, both B^ and B2 a re nonsingular. A similar CQLF existence result has been obtained 
in 



Shorten et al.l ( 20091 ) when one of the matrices (-Bi or B2) is singular. 



We now state a result on the CQLF existence problei n for a pair of matrice s with one of 
them being singular. It is essentially the main theorem in lShorten et al.l ( 20091 ) but we relax 
their assumptions . Let B £ R^ ^ -^ be a real matrix and let g,h £ M . The proposition 
below is stated in IShorten et all (J2009l ) under the assumptions that {B,g) is controllable, 
meaning that the vectors g, Bg, B^g, . . . span M^, and that {B, h) is observable, meaning 
that t he vectors h,B'h, {B')'^h, . . . span M . Using techniques from King and Nathanson 
( 20061 ). we show that these assumptions are unnecessary and we state the result in its full 
generality here. A proof is given in Appendix Rl 

Proposition 3. Suppose that all eigenvalues of matrix B have negative real part and all 
eigenvalues of B — gh' have negative real part, except for a simple zero eigenvalue. Then 
there exists a CQLF for the pair {B, B — gh') if and only if the matrix product B{B — gh') 
has no real negative eigenvalues and a simple zero eigenvalue. 

4 Main results 

In this section, we present our results on positive recurrence of the piecewise OU process 
Y. Key to these results is the following theorem, which uses Proposition [3] to establish the 
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existence of a CQLF for certain matrix pairs. Recall the definitions of i?, p and e from 
Definition [5] in Section 12.11 and note that we are working under Assumption [TJ 

Theorem 1. There exists a CQLF for both the pair {—R, —R{I — pe')) and the pair 
{-R,-{I-pe')R). 

By Theorem [H there exists a CQLF L for the pair {—R,—R{I — pe')) and another 
CQLF L for the pair {—R, —{I — pe')R). Typically there are many CQLFs corresponding 
to these pairs, i.e., L and L are not unique. Note that L and L are closely related in the 
following sense. If the CQLF L for the pair {—R, —R{I — pe')) is given by L(y) = y'Qy for 
some Q > and for all y G M , then one readily checks that the quadratic form L given 
by L{y) = y'{R'QR)y for y G M^ is a CQLF for the pair {-R, -{I - pe')R). We remark 
that, apart from special cases, the CQLFs from Theorem [T] are not explicit. 

We know from Theorem [1] that there exists a CQLF L for the pair {—R, —R{I — pe')), 
where L is given by L{y) = y'Qy for some Q > and for all y G M . We are able to use the 
quadratic form L as a Lyapunov function in the Foster-Lyapunov criterion of Proposition [1] 
to prove the following result. 

Theorem 2. If a = and /3 > 0, then the piecewise OU process Y is positive recurrent 
and has a unique stationary distribution. 

For a > 0, no quadratic function can serve as a Lyapunov function in the Foster- 
Lyapunov criterion to prove positive recurrence of the piecewise OU process Y, see Ap- 
pendix[B]for details. Despite this fact, still relying on TheoremlU we overcome this difficulty 
in Section 15.31 by constructing a suitable non-quadratic Lyapunov function. Specifically, 
there exists a CQLF L for the pair {—R, —{I — pe')R) by Theorem [H where L is given by 
L{y) = y'Qy for some Q > and for all y G M . A suitable approximation to the function 
/, given by for all y G M^, 

f{y) = (e'y)'^ + KL{y — p{e'y)~^) for some large constant k, 

provides the desired non-quadratic Lyapunov function in the Foster-Lyapunov criterion to 
prove positive recurrence of Y when a > 0. Note that, in queueing terminology, the vector 
y — p{e'y)~^ relates to the customers in service, and not to those in the buffer. We therefore 
need the extra term (e'y)^. Applying Proposition [2] with the same non-quadratic Lyapunov 
function yields exponential ergodicity of Y for a > 0. We use a smooth approximation of / 
as a Lyapunov function in the Foster-Lyapunov criterion of Proposition [1] instead of using 
/ directly since / ^ C^(M^'). This leads to the following result. 

Theorem 3. If a > 0, then the piecewise OU process Y is positive recurrent and has a 
unique stationary distribution. Moreover, Y is exponentially ergodic. 
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5 Proof of the main results 

5.1 Proof of Theorem [1] 

Proof of TheoremlJl We only establish the existence of a CQLF for the pair {—R, —R{I — 
pe')), since the existence of a CQLF for the other pair {—R, —{I — pe')R) follows directly. 
Since —R— {—R{I — pe')) = —Rpe' is a rank-one matrix, in view of Proposition [3l we need 
to check three conditions: 

(a) All eigenvalues of — i? have negative real part; 

(6) All eigenvalues of —R{I — pe') have negative real part except for a simple zero eigen- 
value; 

(c) The matrix product R^{I — pe') has no real negative eigenvalues and a simple zero 
eigenvalue. 

We first prove (a) and (b). It is known that all eigenvalues of a nonsingular M- matrix 
have posi tive real part, and all eigenva lues of a singular M-matrix have nonnegative real 
part, see ( Berman and Plemmonsl . ll994l . Chapter 6). Since i? is a nonsingular M-matrix, we 



immediately get (a). For (6), it is clear that —R{I — pe') has a simple zero eigenvalue. We 
notice that {I—pe')R = R—pe'R where e'R > 0' by Assumption[Tl p is a nonnegative vector 
and Ris a nonsingular M-matrix, so the off-diagonal elements of (I —pe')R are nonpositive. 
Using this in conjunction with the fact that both / — pe' and R are M-matrices, we find 
that (/ — pe')R is also an M-m atrix and all its eigenvalues have nonnegative real part, see 
( Berman and Plemmond . Il994l . Exercise 5.2). Thus we get (6) after a similarity transform. 



We now concentrate on proving (c) . The key ingredient of the proof is an identity for 
Chebyshev polynomials. Suppose that R^{I — pe') has a real negative eigenvalue —A with 
A > 0, and write v for the corresponding left eigenvector, thus we have v'R^{I—pe') = —Xv'. 
Right-multiplying by p on both sides, we obtain v'p = and the following equality: 

= v'R^{I - pe') + Xv' = v'R^{I - pe') + Xv'{I - pe') = v'{R^ + XI){I - pe'). (5.1) 

Since i? is a nonsingular M-matrix having only eigenvalues with positive real part, the 
matrix (i?^ -|- XI) is invertible for all A > 0. Also, by the fact that p is a nonnegative 
probability vector with e'p = 1, we deduce the matrix {I — pe') has an eigenvalue and the 
corresponding left eigenvector must be in the form of ce' for some c 7^ 0. Thus, it follows 
from (f5T]) that v' = ce'{R^ + XI)~^ for some c / 0. We show below that e'{R^ + XI)~^ is 
a positive vector for all A > 0, i.e, 

e'[R^ + XI)~^ > 0' for all A > 0. (5.2) 

This yields a contradiction in view of v'p = 0. By definition of a nonsingular M-matrix, R 
is of the form si — N, where A^ is a nonnegative matrix with p{N) < s and e'R > by 
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Assumption [TJ Equation (j5.2p thus states that for ah A > and for every nonnegative 
matrix A^ with p{N) < s and se' > e'N, 

e'{{sI-Nf + XI)-^ >0'. 

Equivalently, we show the fohowing inequahty: for ah y G (0, 1) and for every nonnegative 
matrix N with p{N) < 1 and e' > e'N, 

e'{y{I-Nf + {l-y)I)-'>0'. (5.3) 

Therefore, to show (c), it suffices to prove ()5.3p for fixed N and y G (0, 1). 

Our strategy to prove (|5.3p is to use a matrix series expansion and connections with 
Chebyshev polynomials. Chebyshev polynomials of the second kind Un can be defined by 
the following trigonometric form: 

Un{cos9) = ^^'^(" + 1)^ forn = 0,1,2,3,.... (5.4) 

smt' 

Moreover, for z € [—1, 1] and t G (—1, 1), the generating function of Un is 

oo ^ 

n=0 

Refer to ( Abramowitz and Stegunl . Il992l . Chapter 22) for more details. The scalar version 
of the left-hand side of (|5.3p admits the following expansion: for x,y £ (0, 1), 

^ oo 

Y,Cn{y)x^, (5.6) 



y(l -xf + l-y 



n=0 

where Cn{y) = t^n(v^)(v^)" fo'^ ^^^ n > 0. This can readily be verified with (15. 5|) . In 
particular, we have 

C^{y) = U^{y) = l for aU ye (0,1). (5.7) 

For fixed y G (0, 1), the radius of convergence of the power series in (j5.6p is larger than 1. 
Since p{N) < 1, we immediately obtain that for y G (0, 1), 

oo 

{y{I - Nf + (1 - y)I)-^ = Y, Cn{y)N\ (5.8) 

n=0 

Let y G (0, 1) be fixed and define 9 through ^fy = cos9 G (0, 1). Using the trigonometric 
form (j5.4p of Un, we can then show by induction that for any m > 1, 



Y.^n{y) = J2^nivy)ivvr 



n=l n=l 

m 



y Mn + i)9 

^-^ sm9 

n=l 

^°^[1 - (cos 0)"-^ • COS (m + 1)9] > 0. (5.9) 



sm 



12 



Since A^ is nonnegative and e' > e' N , we immediately get e'A^" > e' N'^^"^ > for all n > 0. 
Combining this fact with (|5.9|) . we obtain 

k k 

e' Y, Cn{y)N'' > Y^ Cn{y)e'N^ > O' for all A: > 1. (5.10) 

n=l n=l 

Therefore, from (|5.7p . (|5.8p and (jS.lUp we conclude that for all y £ (0, 1), 

oo 

e'{{l-y)I + y{I-Nf)~' = e'Y,Cn{y)N^ 

n=0 

k 



= lime'VC„(y)iV" + e' 

n=l 

> 0' + e' = e' > 0'. 



This concludes the proof of (c) and we deduce that there exists a CQLF for the pair 
{-R,-R{I-pe')). 

To prove the existence of a CQLF for the other pair (— i?, —{I — pe')R), we note that 
— (/ — pe')R has the same spectrum as —R{I — pe') and the matrix product R{I — pe')R 
has the same spectrum as R^{I — pe'). Application of Proposition [3] completes the proof of 
Theorem [TJ D 

5.2 Proof of Theorem d 

In this section we prove Theorem [2j Key to the proof is the CQLF constructed from 
Theorem [TJ 

Proof of Theorem\^ If a = 0, then from (j2.3p we know that Y has the piecewise linear 
drift 

b{y) = -Pp - R{y - p{eyy)- 

By Theorem [H there exists a CQLF 

L{y) = y'Qy, (5.11) 

where Q is a positive definite matrix such that 

Q{-R) + i-RyQ<0, (5.12) 

Q{-R{I - pe')) + (-(/ - ep')R')Q < 0. (5.13) 

We claim that given any positive constant C > 0, there exists a constant M > such 
that if \y\ > M, 

{VL{y))'b{y) < -C. (5.14) 
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We discuss the cases e'y < and e'y > separately. 
Case 1: e'y < 0. In this case, we have 

(VL(y))'6(y) = y'lQi-R) + {-R)'Q]y - 2/3p'Qy. 

By (j5.12p . the quadratic term dominates if |y| is large. Thus there exists a constant Mi > 
such that when e'y < and \y\ > Mi, 

{VLiy))'b{y) < -C. 

Case 2: e'y > 0. In this case, we have 

(VL(y))'6(y) = y'[Q{-R{I - pe')) + (-(/ - ep')R')Q\y - 2pp'Qy. (5.15) 

To overcome the difficulty caused by the singularity of —R{I — pe'), we decompose y as 
follows: 

y = ap + ^, (5.16) 

where ^'p = and a G M. Then we have 



\y 
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\ap?' + \i? and e'y = a + e'i>Q. (5.17) 



Note i'ha±p'[Q{-R{I-pe')) + {-{I-ep')R')Q]p = 0. Using (l5J3]) . we obtain p'[Q(-i?(/- 
Ve')) + {-{I - ep')R')Q] = 0'. This immediately implies p'[Q{-R{I - pe')] = 0. Since 
(/ — pe') has a simple zero eigenvalue, we have 

p'Q = be'R--^ for some 6/0. 

Using this fact, we rewrite the left-hand side of (j5.13p as follows: 

Q(_i^(/_pe')) + (_(/_ep')i?')Q 

= ((/ - ep')R') ■ i-QR~^ - {R~^)'Q) ■ (i?(/ - pe')). (5.18) 

After left-multiplying by (i?~^)' and right-multiplying by R~^ in (j5.12p . we deduce that 
[—QR~^ — {R~^)'Q] is a negative definite matrix. Moreover, since ^'p = 0, from (j5.16p and 
()5.18p we know that there exists some c > such that 

y'[Q{-R{I - pe')) + (-(/- ep')R')Q]y 
= y'[{I - ep')R' • {-QR-^ - {R-^)'Q) • R{I - pe')]y 
= e'((/ - ep')R') ■ i-QR-' - {R^'YQ) ■ {R{I - pe'))i 
< -cieP. (5.19) 

Therefore, from (15.15P we have that for any y with e'y > 0, 

{VL{y))'b{y) < -c\^f-2l3p'Q^-2pap'Qp (5.20) 

< -c\^\^-2Pp'Q^ + 2pp'Qpe'^, (5.21) 
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where the second inequahty is obtained from (j5.17|) . /3 > and p'Qp > 0. For \y\ large, if 
1^1 > r for some large constant r, we obtain {SJL{y))'h{y) < —C since the quadratic term 
— c|^P in ()5.2ip dominates. If |^| < r and \y\ large, we deduce from (j5.17p that a must be 
positive and large, i.e., 



\P\ 
Hence the dominating term in (j5.20p is —2(3ap'Qp and we immediately obtain {VL{y)yb{y) < 
—C whenever \y\ is large. Therefore, there exists a constant M2 > such that when e'y > 
and \y\ > M2, 

{VL{y)yh{y) < -C. 

On setting M = max{Mi,M2}, we immediately get (j5.14p . 

Now set C = I Y.i,j Qij{(y'y')i3\ + 1- Equations (|5TT]l and (f5l^ imply that for |y| > M, 

GL{y) = Y,QiM<^% + (VL(2/))'6(y) < -1. 

The proof of Theorem [2] is complete after applying Proposition [TJ D 

5.3 Proof of Theorem [3] 

In this section we prove Theorem [3l Throughout this section, C is a generic positive 
constant which may differ from line to line but is independent of y. 

By Theorem [H there exists a positive definite matrix Q with |(5| = 1 such that 

Q{-R) + {-R)'Q < 0, (5.22) 

Q(-(/ - pe')R) + {-R'{I - ep'))Q < 0. (5.23) 

We construct a non-quadratic Lyapunov function V G C^(M^) as follows. Let 

V{y) = {e'yf + K[y - p<l){e' y)\' Q[y - p^{e'y)l (5.24) 

where k is a positive constant to be decided later and 4>{x) is a real- valued C'^{W) function, 
approximating x 1— )■ x"*". Specifically, fix e > and let 

X if a; > 0, 

— ^e if X < — e, 

smooth if — e < X < 0. 

We piece x > and x < — e together in a smooth way such that cj) is in C^(M), — 2^ ^ 
(^{x) < x"*" and < 0(x) < 1 for any x G M, where (j) is the derivative of (p. This function 
(p evidently exists. Note that V G C^(M^'), but that it is not a CQLF due to its non- 
quadratic nature. We summarize the key result in the following proposition, which implies 
Theorem [3j 
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Proposition 4. If a > 0, there exists a constant C > such that when \y\ is large enough, 
we have 



{VV{y)yb{y) < -C|y|2 and 
Consequently, when \y\ is large. 



d'v 



dyidy. 



(y) 



< C|y| foranyi,j. (5.25) 



GV{y) < -C\y\' < -1. (5.26) 

Proof. We first study {VV{y)yb{y). From (|5.24p . we have for all y G M^^, 

{VV{y)y = 2{e'y)e' + 2K{y' - p' <l,{e' y))Q[I - pe'^{e'y)]. (5.27) 

We discuss the cases e'y > 0, e'y < —e and — e < e'y < separately. 

Case 1: e'y > 0. In this case, let x = e'y and z = y — px = {I — pe')y, then we have 

(VV{y)yb{y) = [2{e'y)e' + 2Ky'(/ - ep')Q{I - pe')]{-R{I - pe')y - ape'y - f3p) 
= -2ax^ - Kz'[Q{I - pe')R + R'{I - ep')Q]z - 2x/3 - 2xe'Rz. 

Suppose we have shown that there exists C > such that 

z'[Q{I - pe')R + R!{I - ep')Q]z > C|zp, (5.28) 

we then obtain that 

{VV{y)yb{y) < -2ax'^ - kC\z\'^ - 2x0 - 2xe'Rz. 

Since a > 0, we can select k > large so that ^{2ax'^ + kCJ^P) > 2\xe'Rz\ for any {x,z), 
where k is independent of (x, z) or y. Then we have, 

(VF(y))'6(y) < -ax^ - -kC\z\^ - 2x/3. 

Note that \y\ = \px + z\ < C|(x,2:)|, so that |(x,z)| is large whenever \y\ is large,. We 
conclude that for \y\ large, 

(VF(y))'6(y) < -C|(x,z)|2 
< -C\y\^- 
It remains to prove (j5.28p . We use a similar argument as for (j5.19p . Observe that 
{R-^pyiQil - pe')R + R!{I - ep')Q]{R~^p) = 0, 
which implies that QR~^p = be for some 6 G M. Thus, we obtain 

z'[Q{I - pe')R + R!{I - ep')Q]z 
= z'R'{I-ep')[{R~^yQ + QR~\l-pe')Rz. (5.29) 
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Since i? is a nonsingular M- matrix, R ^ is a nonnegative matrix ( Berman and Plemmons 
19941 . Chapter 6) and we deduce that 



e'R~'p > 0. 



(5.30) 



This imphes that (/ — pe')Rz ^ since e'z = e'{I — pe')y = in this case. From (j5.22p we 
know that {R~^)'Q + QR~^ is a positive definite matrix. Now (J5.28P fohows from (j5.29p . 
Case 2: e'y < —e. In this case, we have 4>{e'y) = — |e and (l){e'y) = 0. From (|5.22p . 
there exists C > such that 



{VV{y)yb{y) = {2{e'y)e' + 2Ky'Q + Kep'Q){-Ry-l3p) 
= -2k 



y'iQR + R:Q)y + \{ep'QR + Pp'Q)y + ^e^p'Qp 



-2e'y{e'Ry + l3) 



< -2k 



< -2k 



C\y\^ + -{ep'QR + (3p'Q)y + -e(3p'Qp 
C|2/P + \{ep'QR + Pp'Q)y + \el3p'Qp 



- 2e'y ■ {e'Ry + /?) 

+ KC{\y\^ + \y\) 



< -KiC\y\^-C\y\-C), 



where k is again chosen to be independent of y, but large enough such that \2e'y ■ {e'Ry + 
P)\ < KC{\y\'^ + \y\). Thus for \y\ large and e'y < — e, we have 

iVV{y)yb{y) < -C\y\^ 

Case 3: — e < e'y < 0. In this case we use the property that < (j){e'y) < 1. Note 
that we have 

{VViy))'biy) 
= {2{e'y)e' + 2K{y' - p' <t>{e' y))Q{I - pe'^{e'y))){-Ry - pp) 
= 2e'ye'{-Ry-Pp) 

+2K^{e'y){y' - p'<t>{e'y))Q{I -pe'){-Ry - f3p) 

+2k{1 - ^{e'y)){y'-p'<t^{e'y))Q{-Ry - Pp). 

We write 

y = aR~'^p + ^, 

where ^ is orthogonal to R~^p and a G M, so that 

|y|2 = ca^ + |^|2^ for some c> 0. (5.31) 
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From (j5.30p . we have e' R ^p > 0. Without loss of generahty we assume that e'R ^p = 1. 
Then e'y = a + e'^ and we get 

{VV{y)yb{y) 
= -2{a + e'C){l3 + e'RC + a) 

+K^{e'y){^'[Q{-{I -j>e')R) + (-(/ - pe')R)'Q]^ - 2p'Q{I - p^e')R^^{e'y)) 
+k(1 - <j){e'y)){y'[-QR - R'Q]y + (3y'Qp - cP{e'y)p'QRy - p'Qpl3). (5.32) 

Since ^'R~^p = 0, one checks as for ()5.28p that there exists a constant C > such that 

^'[Q(-(/ - pe')R) + (-(/ - pe')RyQ]C < -C\^\^. (5.33) 

Moreover, from ()5.22p and (j5.3ip . we deduce that 

y'[-QR - R'Q]y < -C\y\^ = -Ca^ - C|ep. (5.34) 

Substituting ()5.33p and ()5.34p into (I5.32p . and using < 4>{e'y) < 1 as weh as |0(e'y)| < e, 
we obtain 

{VV{y))'h{y) < -2(a2 + C\a\\i\ + C\a\) + K{-C\i\'^ + C\i\ + C\a\ + C). (5.35) 

Since e'y = o + e'^ G [~£;0]) we must have \a\ < C + \^\ and consequently \y\ < C\a\ + \^\ < 



C|^| + C. Thus for \y\ large, we can choose k large so that the dominating term in (I5.35P 
is — «;C|^p. Using the fact that \y\'^ < C|^p when |y| is large, we then deduce that there 
exists a constant C > such that for \y\ large, 

iVV{y)ybiy) < -C\y\\ 

This concludes the proof for the third case. 

On combining the above three cases we obtain that, for \y\ large, 

(Vy(y))'6(y) < -C\y\^ 

as claimed in the proposition. 

We now proceed to study the second derivative of V, which is denoted by V. We also 
write (j) for the second derivative of (p. From (j5.27p . we find 

V{y) = 2ee' + 2k[Q +^ee' • p'Qp(,^(e'y)<^je'y) + ^{e'yf) 

-{Qpe' + ep'Q)^{e'y) - ee' ■ y'Qp4>{e'y)]. (5.36) 

If e'y ^ [— e,0], we obtain < (t){e'y) < 1 and (j){e'y) = 0. Therefore, for any i,j, there 
exists some C > such that 

<y) < c. 



dyidyj 
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If e'y E [— e, 0], then \(j){e'y)\ < C for some C > since (p £ C^(IR) and [— e, 0] is compact. 
Moreover, since < (j){e'y) < 1, the dominating term in (|5.36p is — 2K.ee' ■y'Qp(j){e'y) for \y\ 
large. This impHes that if e'y G [—£,0] and \y\ is large, then there exists a constant C > 
such that for any i,j, 

-iy) 



<C\y\, 



dyidyj 

where C is independent of y. This concludes the proof of (|5.25p . Now for \y\ large, we 
deduce from ()5.25p that 

1 rl'^V 

GV{y) = {VV{y))'biy) + - ^ (aa'),,——iy) < -C\y\' < -1. 

The proof of Proposition U is complete. D 

Proof of Theorem\^ In order to show that Y is positive recurrent and has a unique station- 
ary distribution, we only have to check that V{y) — )■ oo as |y| — )■ cxd in view of Proposition [1] 
and dOG]) . 

Let X = e'y and z = y — px~^, then |yp < C(x^ + |zp). We can rewrite ()5.24p as follows: 

V{y) = x'^ + K{y' -p'(f>{x))Q{y-p(j){x)) 

> x"^ + C\y - p(l){x)\'^ 

= x'^ + C\z + p{x~^ - (p{x))\'^ 

> x2 + C7|z|2-Ce2 

> C\y\^-Ce^ 

where the second last inequality uses the fact < x^ — (l){x) < ^e. Therefore, V{y) — t- oo 
as \y\ — 7- oo and we conclude that Y has a unique stationary distribution. 

To prove that Y is exponentially ergodic, we observe from (j5.24p that there exists some 
C > such that V{y) < C|yp + C for all y G M.^ . Moreover, (fOGj) implies that for \y\ 

GViy) < -CViy) + C. 

Putting this together with the fact that V G C^(M^), we know that there exist some c > 
and d < oo such that 



GV{y) < -cV{y) + d for any y G 



!,K 



Since V >0, Proposition [2] implies that Y is /-exponentially ergodic, where f = V + 1. In 
particular, Y is exponentially ergodic since / > 1. D 
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A Proof of Proposition [3] 

We first outline the key idea behind the proof. Suppose that {B,g) is not controllable or 
that {B, h) is not observable in the CQLF existence problem. Then we can "reduce" them 
to suitable subspaces such that {Bi,gi) is controllable and {Bi,hi) is observable, where 
Bi is a new matrix of lower dimension than B and similarly for gi,hi. In the process of 
"reduction", two desired properties are preserved: (a) B{B — gh') has no real negative 
eigenvalues if and only if Bi{Bi — gih[) has no real negative eigenvalues; (b) {B, B — gh') 
h as a CQLF if a n d onl y if {Bi,Bi — gih[) has a CQLF. Therefore, applying Theorem 3.1 



m 



Shorten et al.l ( 2003 ) to {Bi,Bi — gih[) yields the result. 



To make the ideas concrete, we now introduce a lemma giving an equivalent formulation 
of the CQLF existence pro blem, which makes th e "red uctioi i" possible. The lemma is a n 
analog of Proposition 2 in King and NathansonI ( 20061 ). In iKing and NathansonI ( 2000 ). 



each matrix of the pair is nonsingular while in our case one of the matrices is singular. 

Lemma 1. Suppose that all eigenvalues of the matrix B have negative real part and all 
eigenvalues of B — gh' have negative real part, except for a simple zero eigenvalue. Then 
the following statements are equivalent: 

(a) The pair {B,B — gh') does not have a CQLF. 

(b) There are positive semidefinite matrices X and Z such that 

BX + XB' + {B- gh')Z + Z{B' - hg') = 0, 
BX + XB' / and {B - gh')Z + Z{B' - hg') / 0. 

(c) There are nonzero, positive semidefinite matrices X and Z such that 

BX + XB' + {B- gh')Z + Z{B' - hg') = 0, (A.l) 

where Z ^ cB^^gg'{B~^)' for any c € M. 

Proof. We first prove the equivalence of (a) and {b). To set up the notation, let S^^^ be 
the space of real symmetric K x K matrices. For an arbitrary matrix A G M , define 
the linear operator La on S^^^ by 

La-.S^^'^^S^''^, La{H) = AH + HA'. (A.2) 

It is well-known that if A has eigenvalues {Aj} with eigenvectors {vi}, then La has eigen- 
values {Aj -|- Aj} with eigenvectors {viv'j + Vjv'j] for all i < j. Since all eigenvalues of the 
matrix B hav e negative real part, Lp is i nvertible. 

Following iKing and NathansonI ( 20061 ). we formulate the CQLF existence problem in 



terms of separating convex cones in S^^^. Define Cone{B) = {Lb{X)\X > 0} and 
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Cone{B-gh') = {Lt^B-gh'){Z)\Z > 0}. Both are closed convex cones in 5^^^. Let S^""^ 
be equipped with the usual Hilbert-Schmidt inner product {X,Z) = tic{XZ). We obtain 
that for any Q G 5^^^, 

{X, QB + B'Q) = {Q, BX + XB') = {Q, Lb{X)). 

Note that for a nonzero positive semidefinite matrix X, we have QB+B'Q < if and only if 
(X, QB + B'Q) < 0, where the "if" part can be checked by taking X = xx' for any nonzero 
X S M , and the "only if" part follows from the spectral decomposition of the positive 
semidefinite matrix X. Therefore, we have QB + B'Q < if and only if {Q, M) < for all 
nonzero M £ Cone(i?). Using a similar argument one finds that Q{B—gh') + {B—hg')Q < 
if and only if {Q,T) < for all nonzero T £ Cone(i? — gh'). Moreover, since B only has 
eigenvalues with negative real part, we deduce that QB + B'Q < for Q S gKxK ^j^p^jgg 
that Q is positive definite by Theorem 2.2.3 in lHorn and JohnsonI ( 19941 ). By definition of 



CQLF, we thus obtain that {B, B — gh') has a CQLF if and only if there exists a Q G gKxK 
such that QB + B'Q < and Q{B - gh') + {B - hg')Q < 0. Equivalently, {B, B - gh') 
has a CQLF if and only if there exists a Q G qKxK g^^j-^ i}xsA. {Q, M) > for all nonzero 
M € Cone(— B) and {Q,T) < for all nonzero T S Cone(i? — gh'). Therefore, finding a 
CQLF for the pair {B, B — gh') is the same as finding a separating hyperplane in S^^^ for 
Cone(— i?) and Cone(S — gh'). By the separating hyperplane theorem, we conclude that 
{B,B — gh') not having a CQLF is equivalent to Cone(— i?) and Cone(-B — gh') having 
nonzero intersection. This completes the proof of the equivalence of (a) and (b). 

We now turn to the equivalence of (6) and (c), for which we use the aforementioned 
spectral properties of the linear operator ()A.2p . Since Lb is invertible, we deduce that 
Lb{X) = is equivalent to X = 0. We know that all eigenvalues of {B — gh') have 
negative real part except for a simple zero eigenvalue, hence LiB-gW) ^-Iso has a simple 
zero eigenvalue with eigenvector cB~^gg'{B~^)' for some nonzero c € M while all of its 
other eigenvalues have negative real part. Consequently, (B — gh')Z + Z[B — gh')' 7^ is 
equivalent to Z ^ cB^^gg'{B~^)' for any c G M. The proof of the lemma is complete. D 



Proof of Proposition\^ In view of Theorem 3.1 of lShorten et al.l ( 20091 ). we need to check 



that controllability of {B, g) and observability of (S, h) need not be verified in the CQLF ex- 
istence problem. Recall that controllability of {B, g) means that the vectors g, Bg, B^g, . . . 
span M^, and observability of {B, h) means that the vectors h, B'h, [B')^h, . . . span M^. 
To simplify the notation, \et B = B — gh' . 

We first show that in the CQLF existence problem for the pair {B,B — gh'), we can 
assume without loss of generality that {B, g) is controllable . The proof relies on Lemma [TJ 
Let U be the span of vectors g, Bg, B^g .... Suppose C/ is a proper subspace of M^ with 
dini{U) < K, and note that M^ = C/ © C/-*- where C/-*- is the orthogonal complement of U. 
In view of this decomposition, we perform a change of basis and rewrite B, B, X and Y 
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in the block form 

where B — B = gh' and g, h are represented in the new basis. We use the same notation 
for the matrices and vectors after the change of basis to save space, and we remark that 
the orthogonal transformation does not affect the existence of a CQLF for the pair [B, B) 
or the existence of real negative eigenvalues of BB. Namely, for any orthonormal matrix 
O G M , one readily checks that the pair {B,B) has a CQLF if and only if the pair 
{OBO', OBO') has a CQLF. Furthermore, BB has no real negative eigenvalues if and 
only if (OBO'){OBO') has no real negative eigenvalues. Let gi,hi be the orthogonal 
projection of g, h on the subspace U, so that Bi — Bi = gih[. Since U is the span of the 
vectors g,Bg,B^g . . ., we deduce that gi,Bigi,Bfgi . . . span U by (IA.3h . i.e., {Bi,gi) is 
controllable. We now use Lemma [1] to argue that there exists a CQLF for {B,B) if and 
only if there exists a CQLF for {Bi,Bi), where {Bi,gi) is controllable. Note that (jA.3p 
implies, using (jA.ip in Lemma [H 

Bs{Xs + Zs) + (X3 + Zs)B!, = 0. 

Equivalently, 

Lb,{X3 + Z3)=0, 

where the linear operator Lb-j is defined in (IA.2p . Since B has only eigenvalues with nega- 
tive real part, S3 also has this property. This implies the linear operator Lb^ is invertible. 
We thus obtain X3 + Z3 = 0. Using the fact that X and Z are positive semidefinite, we 
deduce that X3 = Z3 = 0, and consequently X2 = Z2 = 0- This leads to 

BiXi + XiB[ + BiZi + ZiB[ = 0. (A.4) 

Thus for the pair {B,B — gh'), the existence of nonzero X,Z > such that (jA.ip holds 
implies the existence of nonzero Xi,Zi > such that (1A.4P holds. Conversely, if there 
exists nonzero X\,Z\ > such that ()A.4p holds, setting X2 = X3 = Z2 = Z-^ = 0, we 
then obtain that there exists nonzero X,Z > such that (jA.ip holds. Since B — gh' 
has only eigenvalues with negative real part except for a simple zero eigenvalue, so does 
Bi - gih'^. For c G M, since g e U, one finds that g'{B~^)' = (c/i(Sf ^)', 0') by fOl) . Thus 
Z / cB^^gg'{B~^y is equivalent to Zi / cB^^gig'^{B^^)' . Putting these facts together, 
we apply Lemma [1] to conclude that {B,B) has no CQLF if and only if (Bi,Bi) has no 
CQLF, where {Bi,gi) is controllable. Therefore, without loss of generality, we can assume 
that {B,g) is controllable in the CQLF existence problem for the pair {B, B — gh') . 

We next show that without loss of generality we can assume that {B, h) is observable 
in the CQLF existence problem for the pair {B,B — gh'). Note that for Q > 0, we have 
QB + B'Q < and Q{B - gh') + {B' - hg')Q < if and only if Q-'^B' + BQ'^ < 
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and Q-\B - hg') + {B' - gh')Q''^ < 0. Hence {B,B - gh') has a CQLF if and only if 
(S', B' — hg') has a CQLF. From the preceding paragraph, we know that in the CQLF 
existence problem for the pair (B',B' — hg'), we can assume that {B',h) is controllable 
without loss of generality. By definition, {B',h) being controllable is the same as {B,h) 
being observable. Therefore, we conclude that we can assume without loss of generality 
that (B, h) is observable. 

Finally, we argue that the pair {B, B — gh') has a CQLF if and only if the matrix 
product B{B — gh') has no real negative eige nvalues. Assuming t hat {B,g) is controllable 



and that {B, h) is observable. Theorem 3.1 in [Shorten et al.l (J2009 ) states that [B, B — gh' 



has a CQLF if and only if the matrix product B{B — gh') has no real negative eigenvalues. 
We have shown that we can always assume that {B,g) is controllable and that {B,h) is 
observable in the CQLF existence problem by reduction to proper subspaces. So it only 
remains to check that in the process of reduction, the spectral property of having no real 
negative eigenvalues of the matrix product is preserved. Specifically, in the above proof 
that controllability of {B,g) can be assumed without loss of generality, we obtain that 
{B,B — gh') has a CQLF if and only if {Bi,Bi — gih'^) has a CQLF, where {Bi,gi) is 
controllable. We next prove that B{B — gh') has no real negative eigenvalues if and only 
if Bi{Bi — gih'i) has no real negative eigenvalues, i.e., the desired spectral property of the 
matrix product is preserved in the process of reduction from (B,B — gh') to {Bi,Bi—gih'i). 
Observe that the spectrum of B[B — gh') is the union of the spectrum of Bi{Bi — gih'^) 
and i?| by (|A.3p . Since all eigenvalues of B^ have negative real part, we deduce that 
Bi{Bi — gih'i) having no real negative eigenvalues is equivalent to B(B — gh') having no real 
negative eigenvalues. A similar argument applies for observability instead of controllability. 
We have therefore completed the proof of Proposition [3l D 

B Any quadratic function fails for a > 

In this section, we give a simple example showing that, in general, no quadratic function can 
serve as a Lyapunov function in the Foster-Lyapunov criterion to prove positive recurrence 
of the piecewise OU process Y for a > 0. We first introduce a lemma which implies that 
the matrix —R{I — pe') — ape' is nonsingular for a > 0. 

Lemma 2. If a > 0, then all eigenvalues of the matrix —R{I — pe') — ape' have negative 
real part. 

Proof. It is clear that the matrix has an eigenvalue —a with right eigenvector p. Suppose 
A ^ —a is an eigenvalue of the matrix with left eigenvector 9, i.e., 

e'{-R{I - pe') - ape') = X9', (B.l) 

then we obtain that 9'p = 0. It follows from (jB.ip that A is an eigenvalue of the matrix 
—R{I — pe'). Moreover, A cannot be zero since otherwise 9' = ce'R~^ for some nonzero 
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c G M, which follows from the fact that R{I — pe') has a simple zero eigenvalue. This 
contradicts the fact that e'R~^p > as seen in (|5.3U|) . From condition (6) in the proof of 
Theorem [H we know that all nonzero eigenvalues of the matrix —R[I — pe') have negative 
real part. This completes the proof of the lemma. D 

Lemma 3. Suppose that Q is a real K x K positive semidefinite matrix such that at least 
one of the matrices Q{—R) + {—R')Q and Q{—R(I — pe') — ape') + (—(/ — ep')R' — aep')Q 
fails to be negative definite. Let the quadratic function L be given by L{y) = y'Qy for 
y G M . Then there exists some /3 € M and v G M such that GL{tv) > for any i > 0. 

Proof. Suppose that Q{—R) + {—R')Q fails to be negative definite, then there exists some 
A > and nonzero vector v € K^ such that [Q{—R) + {—R')Q]v = Xv and e'v < 0. By 
definition of generator of Y in (j2.2p . we thus obtain 



GLitv) = Y,Qij{aa')ij + {VL{tv))'b{tv) 

= Yl Qij{aa')ij + t^v'[Q{-R) + i-R')Q]v - 2t/3p'Qv 

= YQij{aa')ij + \v'vt^ -2tl3p'Qv. (B.2) 

Since Q is positive semidefinite, we infer that ^^ • Qij{cra')ij = tr {Qcra') = tr (a' Qa) > 0. 
Set /3 = 0. We conclude from ()B.2p that GL{tv) > for any t > 0. A similar argument 
applies to the case where Q{— R{I — pe') — ape') + {—{I — ep')R' — aep')Q fails to be negative 
definite. The proof of the lemma is complete. D 

In view of Lemmas [2] and [3l we give the following definition of strong CQLF which 
is slightly different th a ,n De finiti on [5] given in Section 13. 1[ For more details, refer to 



Shorten and Narendral ( 20031 ) and King and NathansonI (120061 ) 



Definition 6 (strong CQLF). Let A and B be real KxK matrices having only eigenvalues 
with negative real part. For Q € R"- ^"- , the quadratic form L given by L{y) = y'Qy for 
y S M^ is called a strong common quadratic Lyapunov function (strong CQLF) for the 
pair {A, B) if Q is positive definite and 

QA + A'Q < 0, 
QB + B'Q <0. 

We remark that i t suffices to require Q to b e a symmetric matrix in the above definition 



by Theorem 2.2.3 in lHorn and JohnsonI (jl994l ) 



We now formulate an example showing that, in general, no quadratic function can serve 
as a Lyapunov function in the Foster-Lyapunov criterion to prove positive recurrence of 
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the piecewise OU process Y for a > 0. Let i? be a matrix given by 



R 



so that R is a, nonsingular M-matrix. Let a = 133 and p' = [0, 0, 1]. 

Lemma 4. For any quadratic function L given by L{y) = y'Qy for some real K x K 
positive semidefinite matrix Q and all y G M^, there exists some /? E M and v G M^ such 
that GL{tv) > for any t &M. in the above example. 

Proof. Li view of Lemma [3l it suffices to prove that there is no strong CQLF for the pair 
{—R, — R{I — pe') — ape') for a > 0. Equivalently, it suffices to show th at the matrix product 



R(R(I — pe') + ape') has real negative eigenvalues by Theorem 1 in lKing and Nathanson 



( 20061 ). One readily checks that R{R{I — pe') + ape') has three different eigenvalues: —7 



5 — \/82 and 5 + -v/82- Thus, it has two real negative eigenvalues and we deduce that 
{—R, —R{I — pe') — ape') has no strong CQLF in this example. Application of Lemma [3] 
completes the proof of the lemma. D 
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